Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test auto removal #345

Merged
merged 8 commits into from
Sep 19, 2023
Merged

Test auto removal #345

merged 8 commits into from
Sep 19, 2023

Conversation

adam-mateen
Copy link
Contributor

@adam-mateen adam-mateen commented Sep 15, 2023 •
edited
Loading

Description of the issue

A customer had concerns over the behavior of the agent's auto_removal feature.
They reported the agent was deleting their file BEFORE they created a 2nd file matching the pattern.
This test just verifies the agent DOES NOT delete the monitored file, even if it is restarted after reading to the EOF.

And I added another test case that verifies the agent DOES DELETE prev_file when new_file matching the pattern is found.

Description of changes

Adds a new test case.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

  • SSH'd into an EC2 instance and run the test from shell:
[ec2-user@ip-172-31-18-102 ~]$ cd go/src/github.com/aws/amazon-cloudwatch-agent-test/
[ec2-user@ip-172-31-18-102 amazon-cloudwatch-agent-test]$ go test -v -p=1 -count=1  ./test/cloudwatchlogs/... -computeType=EC2 -run TestAutoRemovalFileRotation
=== RUN   TestAutoRemovalFileRotation
2023/09/15 19:06:36 Testing all EC2 plugins
2023/09/15 19:06:36 Testing all EC2 plugins
2023/09/15 19:06:36 Starting agent with command sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:resources/config_auto_removal.json
2023/09/15 19:06:37 Agent has started
2023/09/15 19:06:57 Writing 2000 lines to /tmp/cwagent_log_test.log0
2023/09/15 19:07:39 Writing 2000 lines to /tmp/cwagent_log_test.log1
2023/09/15 19:08:20 Writing 2000 lines to /tmp/cwagent_log_test.log2
2023/09/15 19:09:01 Writing 2000 lines to /tmp/cwagent_log_test.log3
2023/09/15 19:09:42 Writing 2000 lines to /tmp/cwagent_log_test.log4
2023/09/15 19:11:03 Checking i-0fc7939c1328a611e/i-0fc7939c1328a611e
2023/09/15 19:11:04 Done paginating log events for i-0fc7939c1328a611e/i-0fc7939c1328a611e and found 10000 logs
--- PASS: TestAutoRemovalFileRotation (268.16s)
PASS
ok  	github.com/aws/amazon-cloudwatch-agent-test/test/cloudwatchlogs	268.162s

And

[ec2-user@ip-172-31-18-102 amazon-cloudwatch-agent-test]$ go test -v -p=1 -count=1  ./test/cloudwatchlogs/... -computeType=EC2 -run TestAutoRemovalStopAgent
=== RUN   TestAutoRemovalStopAgent
2023/09/15 19:13:50 Testing all EC2 plugins
2023/09/15 19:13:50 Testing all EC2 plugins
2023/09/15 19:13:50 Starting agent with command sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:resources/config_auto_removal.json
2023/09/15 19:13:52 Agent has started
2023/09/15 19:14:12 Writing 2000 lines to /tmp/cwagent_log_test.log1
2023/09/15 19:14:33 Agent is stopped
2023/09/15 19:14:33 Testing all EC2 plugins
2023/09/15 19:14:33 Testing all EC2 plugins
2023/09/15 19:14:33 Starting agent with command sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:resources/config_auto_removal.json
2023/09/15 19:14:34 Agent has started
2023/09/15 19:14:54 Writing 2000 lines to /tmp/cwagent_log_test.log1
2023/09/15 19:15:16 Agent is stopped
2023/09/15 19:15:16 Testing all EC2 plugins
2023/09/15 19:15:16 Testing all EC2 plugins
2023/09/15 19:15:16 Starting agent with command sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:resources/config_auto_removal.json
2023/09/15 19:15:17 Agent has started
2023/09/15 19:15:37 Writing 2000 lines to /tmp/cwagent_log_test.log1
2023/09/15 19:15:58 Agent is stopped
2023/09/15 19:15:58 Testing all EC2 plugins
2023/09/15 19:15:58 Testing all EC2 plugins
2023/09/15 19:15:58 Starting agent with command sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:resources/config_auto_removal.json
2023/09/15 19:16:00 Agent has started
2023/09/15 19:16:20 Writing 2000 lines to /tmp/cwagent_log_test.log1
2023/09/15 19:16:41 Agent is stopped
2023/09/15 19:16:41 Testing all EC2 plugins
2023/09/15 19:16:41 Testing all EC2 plugins
2023/09/15 19:16:41 Starting agent with command sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:resources/config_auto_removal.json
2023/09/15 19:16:42 Agent has started
2023/09/15 19:17:02 Writing 2000 lines to /tmp/cwagent_log_test.log1
2023/09/15 19:17:24 Agent is stopped
2023/09/15 19:18:24 Checking i-0fc7939c1328a611e/i-0fc7939c1328a611e
2023/09/15 19:18:24 Done paginating log events for i-0fc7939c1328a611e/i-0fc7939c1328a611e and found 10000 logs
--- PASS: TestAutoRemovalStopAgent (274.21s)
PASS

@adam-mateen adam-mateen requested a review from a team as a code owner September 15, 2023 13:17
straussb
straussb previously approved these changes Sep 15, 2023
View reviewed changes
test/cloudwatchlogs/publish_logs_test.go
loopCount := 5
linesPerLoop := 1000
start := time.Now()
for i := 0; i < loopCount; i++ {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I see the need to do this repeatedly in a loop, but doesn't hurt.

@adam-mateen adam-mateen merged commit 0ec07d2 into main Sep 19, 2023
2 checks passed
@adam-mateen adam-mateen deleted the test_auto_removal branch September 19, 2023 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JayPolanco JayPolanco JayPolanco approved these changes

@ChenaLee ChenaLee ChenaLee approved these changes

@straussb straussb straussb left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@adam-mateen @ChenaLee @JayPolanco @straussb